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Abstract 

In  this  paper,  optimal  conurol  problems 
where  there  is  uncertainty  in  the  initial  state 
measurement  or  where  there  are  uncertain 
parameters  in  the  state  equations  are  investi- 
gated. It  is  assumed  that  nature  will  choose 
the  uncertainty  to  maximize  the  cost  which  the 
controller  is  attempting  to  minimize.  Thus  a 
minmax  control  is  sought. 

Sufficient  conditions  for  a control  to  be 
a minmax  control  are  presented.  These  condi- 
tions suggest  methods  for  finding  the  minmax 
control  and  such  techniques  are  described.  The 
application  of  these  methods  is  illustrated  by 
example  problems. 


1.  Introduction 

In  an  optimal  control  problem,  only  uncertain 
measurements  of  the  initial  stale  may  be  available 
rather  than  knowledge  of  the  exact  initial  state. 
One  approach  to  this  problem  is  to  obtain  a sto- 
chastic description  of  the  uncertainty  and  choose 
the  control  to  minimize  an  expected  value.  Here 
the  problem  is  treated  in  a different  fashion.  It 
is  assumed  that  from  the  measured  initial  state  it 
is  only  possible  to  conclude  that  the  true  in* Mai 
state  belongs  to  some  subset  of  the  state  space. 

The  objective  is  to  choose  a control,  based  on  this 
measurement,  which  minimizes  the  maximum  value  of 
the  cost  over  all  possible  initial  states  in  the 
subset.  Thus  it  is  assumed  that  nature  is  perverse 
and  chooses  the  uncertainty  to  maximize  the  cost 
which  the  controller  is  attempting  to  minimize. 

For  each  control  there  is  a guaranteed  performance 
(which  is  determined  by  assuming  nature  maximizes 
against  this  control)  and  the  optimal  control  is 
the  one  which  achieves  the  best  guaranteed  per- 
formance . 

If  Q is  the  set  of  possible  values  for  the 
uncertainty,  ft  the  set  of  admissible  controls  and 
J(u(*),q)  the  cost  functional,  then  the  problem  is 

* 

to  find  a u (•)  sift  satisfying  for  all  u(-)  ern 
★ 

sup  j(u  (-),q)  s sup  J(u(-),q) 
q£Q  qeQ 

The  first  sufficient  condition,  Theorem  1,  is 

^he  research  was  supported  by  the  Air  Force  Office 
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applicable  when  the  cost  function  has  a saddle 
point.  However,  examples  show  that  often  this  is 
not  the  case  and  the  second  sufficient  condition. 
Theorem  2,  can  be  used  when  there  is  no  saddle 
point  solution. 

In  obtaining  the  second  sufficient  condition, 
the  initial  state  uncertainty  problem  is  trans- 
formed into  a problem  with  known  initial  state  but 
having  uncertainty  in  the  state  equations.  Thus 
the  result  is  also  applicable  to  problems  where 
the  mathematical  model  contains  uncertainty  ir.  the 
differential  equation  describing  the  evolution  n 
the  state.  Both  sufficient  conditions  suggest  a 
method  for  constructing  a minmax  control  and  these 
procedures  are  described. 

Initial  state  uncertainty  problems  have  been 
studied  in  [l]  where  a general  result  for  the 
linear  quadratic  case  is  obtained.  Our  results 
are  not  limited  in  application  to  such  problems. 
The  minmax  approach  to  uncertainty  has  also  been 
investigated  in  [2]-[8],  The  results  presented 
here  are  different  and  appear  to  be  applicable  to 
a wider  class  of  problems  than  those  considered 
in  most  of  these  papers  since  the  assumption  of 
the  existence  of  a saddle  point  solution  is  not 
required  in  Theorem  2 nor  are  the  results  limited 
to  linear  quadratic  problems. 

2.  Problem  Formulation 

Consider  a system  which  can  be  modeled  by 
ordinary  differential  equations 

*(t)  = f(x(t),u(t)),  te[to,tf]  (!) 

where  x(t)  € Rn  is  the  state,  u(t)  e Rm  is  the  con- 
trol and  the  time  interval  ft  ,t.J  is  prescribed. 

of 

A measurement  oi  the  initial  istjtc,  v , is  avail- 

otu 

able  and  is  related  to  the  true  initial  state, 
x(to),  by 

x(t  ) = x + q (21 

o om  ^ 

where  q€QCRn  and  Q is  known.  If  q were  known 
exactly,  then  we  would  have  the  usual  optimal  con- 
trol problem.  Here,  however,  we  assume  that  q is 
not  known  exactly  but  is  chosen  perversely  by 
nature . 

A control  u(*)  will  be  called  admissible  if 
it  is  piecewise  continuous  and  u(t>  eU  for  all 

te[to,t^]  where  UcRm  is  a given  set.  The  set  of 
admissible  controls  will  be  denoted  by  fU.  We 
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shall  assume  throughout  that  for  every  u(-)  €lU  and 
qsQ  there  is  a solution  of  (1)  and  (2)  on  [t^t^]. 

The  cost  or  criterion  depends  on  the  choice  of  the 
control  u ( • ) and  the  parameter  q. 


d(u(*),q)  = ®(x(t,))  + I L(x(t) ,u(t))dt  (3) 
"t 

o 

* The  problem  is  to  find  the  optimal  control, 

u (•),  based  on  the  measurement  x when  the  opti- 

om 

mality  criterion  is  minmax,  i.e.,  find  an  admissi- 

* 

ble  control  u (•)  satisfying 
★ 

sup  J(u  (• ) ,q)  £ sup  J(u(-),q)  (4) 

qeQ  qeQ 

for  all  u(.)  cfR. 

One  approach  to  the  problem  would  be  to  deter- 
mine q(u(-))  satisfying  for  every  admissible  u(‘) 

J(u(>)  ,q(u(-)))  * j(u(-),q)  V qeQ 

and  then  determine  the  admissible  control  which 
minimizes  J(u(* ) ,q (u(- ) ) ) • This  approach,  however, 
is  not  feasible  because  of  the  difficulty  in  deter- 
mining q (u  ( • ) ) . 

Alternatively,  one  can  assume  q is  fixed  and 
find  u(.,q)  satisfying  for  all  qeQ 

J(u(’,q).q)  4 J(u(.),q)  v u(.)  «m 

If  q° e Q maximizes  J(u(',q),q),  thenu(*,q°)  is  a 
candidate  for  the  minmax  control.  While  it  may  be 
possible  to  perform  the  above  steps,  the  resulting 

control,  u ( • , q° ) , can  only  be  the  minmax  control  if 
J(u(-),q)  has  a saddle  point  solution,  i.e.,  there 

is  a (u°(»),q°)  satisfying 

J(u°( • ) ,q)  £ J(u°(.),q°)  4 J(u(.),q°) 

for  all  u(»)  Cm  and  qeQ.  This  will  not  always  be 
the  case  for  these  initial  state  uncertainty  prob- 
lems. Judging  from  the  examples,  what  will  usually 

occur  in  a problem  is  that  for  some  values  of  x 

r om 

there  will  be  a saddle  point  solution  while  for 
others  there  will  not. 

In  the  next  section,  we  present  a sufficient 
condition  for  a minmax  control  when  there  is  a 
saddle  point  solution  and  in  Sec.  5 a sufficient 
condition  which  applies  when  a saddle  point  solu- 
tion does  not  exist.  In  both  cases,  the  sufficient 
conditions  suggest  a constructive  method  for  find- 
ing the  minmax  control. 

In  the  following, we  assume  f(»,»)  :RnxRm-*Rn, 
0 ( • ) : Rn  -*  R and  L( • , • ) : R°  x Rm  - R are  C * func - 
tions  and  that  Q * (q  : C(q)  £ 0}  where  C(«)  : Rn-»RP 
is  also  C . 


3.  The  First  Sufficient  Condition 


Theorem  1.  Suppose  0(«)  is  convex  on  R . Let 

u (•)  be  admissible  and  h(q)  » J(u  (-),q).  If 

* 

there  exists  a q IQ  and  a non-positive  vector  p 
such  that 


id 


&C1(q  ) 


I - {i  : Ct(q  ) - 0} 


11)  (h(q)  + r P^q))  £ 0 V qeRn 

3q  lex 

and  if  there  exists  an  absolutely  continuous* 

n 9«(x  (tf)) 

function  g(')  : [t  , t ] - R with  g(t.)  r- 

. ..  . of  t 

such  that 

iii)  A = L(y,v)  + gT(t)f(y,v)  - L(x  (t)  ,u  (t)) 

- gT(t)f(x*(t),u*(t))  + BT(t)(y-x*(t))*0 

V yeRn,  V v€U  and  for  almost  all  tc[.t  ,t_J 
* ° f 
where  x (t)  is  the  solution  of 

x (t)  = f (x*(t)  ,u*(t)) , x (tQ)  - x^+q 

* 

then  u (•)  is  a minmax  control. 

Proof.  Conditions  (i)  and  (ii)  imply  h(q)  £ 

h (q  ) V qCQ  or  J(u(  (’),q)  4 J(u  (0>q  ) v qtQ. 
Following  [9],  condition  (iii)  implies 

J(u  (• ) ,q  ) £ J(u(-),q  ).  Thus  (u  (-),q  ) is  a 
saddle  point  solution  and,  since  all  pairs 

* 

(u(«)*q)  with  u(«)  cm  and  qeQ  are  playable,  u (•) 
is  a minmax  control.  O 

Condition  (iii)  of  Theorem  1 is  used  to  show 

that  J(u  (•)  ,q  ) £ J(u(*),q  ) for  all  u(-)  cITl. 
Rather  than  using  a simple  sufficiency  approach 
one  could  use  a field  theorem  such  as  [lO,ll]  to 

show  that  J(u  (O.q  ) £ J(u(’),q  )•  If  this  is 
done,  the  assumption  that  $(•)  is  convex  can  be 
dropped. 

This  theorem  suggests  the  following  procedure 
for  finding  a minmax  control. 

1.  Solve  the  necessary  conditions  for  the  optimal 
control  problem  ( 1) - (3)  assuming  q is  known. 
This  yields  u(- ,q) . 

2.  Evaluate  J(u(-,q),q), 

3.  Maximize  J(u(-,q),q)  subject  to  C(q)  £ 0. 

Call  the  maximizing  solution  q°. 

* * o 

4.  Let  u (•)  * u(*,q  ) and  evaluate  h(q)  * 

J(u  (• ) *q) • 

5.  Check  the  sufficient  condition  of  Theorem  1 

ie 

with  u (•)  as  the  candidate.  This  involves 
★ 

finding  a q and  p satisfying  (i)  and  (ii). 

In  (iii),  B(-)  can  be  taken  as  the  multiplier 
from  the  optimal  control  problem  of  Step  1 
* 

with  q = q . 

This  procedure  often  works  when  there  is  a 

saddle  point  solution  for  the  particular  x under 

om 

consideration  but  will  not  work  when  there  is  no 
saddle  point  solution.  A simple  example  illus- 
trates this  technique. 

1 

J(uO).q)  - | X2(l)  + -|  J u2(t)dt  (5) 
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o: 


owr 


x(t)  - u ( t ) , X(tQ) 


x + q 
om  ^ 


Q - tq  : lq|  * lj 


From  Step  1 we  obtain  u(t,q)  » - — (x  + q)  and 

. Z om 

from  Step  2 J(u(*,q),q)  - 7 (x  + q)^.  The 

4 om  . 


-T  * 

adjoint  variable  in  Step  1 la  \(t)  - 7 (x  + q) . 

z om_ 


The  maximization  problem  In  Step  3 yields  q - 1 
if  x 20  and  q°  ■ -1  if  x < 0.  Then  u (t)  - 


this  form,  the  condition  will  also  be  a sufficient 
condition  for  the  initial  state  problem.  Unlike 
Theorem  1,  this  sufficient  condition  is  applicable 
when  there  is  no  saddle  point  solution.  Of  course, 
the  sufficient  condition  also  applies  to  problems 
which  can  be  formulated  as  (8) -(10)  and  we  also 
have  a sufficient  condition  for  optimal  control 
problems  where  there  is  parameter  uncertainty  in 
the  state  equations.  Results  for  problems  with 
time  varying  uncertainty  in  the  state  equations 
are  given  in  [12,13]. 


u(t,q  ) • - +1)  if  x *0  while  u (t) 

z oro  om 


5.  The  Second  Sufficient  Condition 


■kx  -1)  if  * <0.  Also  h(q)-£(x  +1)2  + 

Z om  om  4 om 


i(x  +q)2  -|(x  +l)(x  +q)  if  x 20  and  h(q) 

Z om  Z om  om  om 


f(x  -l)2  + -i(x  +q)2-i(x  -l)(x  +q)  if  X <0. 

4 om  z om  Z om  om  om 


If  x 21  the  sufficient  condition  of 
om  . 


Theorem  1 are  satisfied  with  the  above  u (•)  and 
q*  - +1,  p - - \ (x  +1),  8 (t)  - (x  +1)  while 


Consider  the  problem  (8)-(10). 

Theorem  2.  Suppose  i|i(’>q)  i«  a convex  function 

n * . 

of  z on  R for  all  q and  let  u (•)  em.  If  there 

exists 

a)  a positive  Integer  Y 

b)  vectors  q , i - 1,«*»,Y 

c)  absolutely  continuous  functions 


11  ” 4 om  ’ ' ' 2 ' om' 

if  x s -1,  they  are  satisfied  with  u*(-)  and 
* 0111  1 1 

q - -1,  P - 7 <*  -D,P(t)  - k (x  - 1).  For 


xV)  : [t  ,t  ] - Rn,  i - 1,2,  • • • ,Y 

Y 


-1  < x <1,  the  sufficient  conditions  cannot  be 
om 


satisfied  and  we  suspect  that  there  is  no  saddle 
point  solution  when  -1  < x < 1, ' In  the  next 


d)  scalars  or  >0,  i - 1,**»,Y  with  Z a , - 1 

1 1-1  1 
such  that  . . . 

, iiKz1  (t.).q1) 

i)  Xl(te)  - «1 jjl ,i-l,...,Y 


sections  we  present  a method  for  treating  such 
situations. 


4.  A Transformation 


Me  shall  derive  sufficient  conditions  for 
mlnmax  control  when  there  is  no  saddle  point  solu- 
tion by  considering  an  equivalent  problem. 

Let  z(t)  - x(t)-q  and 


where  z (•)  is  the  trajectory  corresponding 
to  (u*(-).ql) 

ii)  q1*£(u*(-))>  i “ 1, • • • ,Y  where  £(u  (•))  - 

[q  : qcQ  and  J(u  (.),q)  " «up  J(u  (O.p)} 
pcQ 

V V 

iii)  ^ cr.M(yi,w,q1)  + Z Xi(t)k(y1,w,qi) 

i-1  1 1-1 


k(z(t),u(t),q)  - f (z (t)  + q,u(t) ) 
♦ - 0(z(tf)  + q) 

M(z(t) ,u(t) ,q)  - L(z(t)  + q,u(t)) 


Z Of  M(z1*(t)  ,u*(t)  .q1) 
i-1  1 


- Z Xi(t)k(z1*(t),u*(t),qi) 


Now  consider  the  optimal  control  problem 


z(t)  - 

4 

k(z(t) , u (t) ,q) 

(8) 

+ 

y 

z Xi(t)(yi-Z1*(t))  2 0 

N 

rr 

O 

■ X 

om 

rtf 

(9) 

V 

i-1 

yieRn,  i-l,2,...,Y, 

1 K(u(*) 

A 

,q)  - ♦ (*(tf) ,q)  + J M(z(t) ,u(t) ,q)dt 

(10) 

V 

wCU  and  for  almost  all  te[t  ,t,] 

0 z 

The  original  problem  has  been  transformed  from 
one  with  initial  state  uncertainty  to  one  with  ini- 
tial state  known,  but  with  an  uncertain  parameter 
in  the  state  equations  and  cost.  For  any  u(<)  cth 

and  qeQ,  K(u(*),q)  • J(u(-),q).  Thus  if  u*(*)  is 

a mlnmax  solution  to  (l)-(3)  then  u (•)  is  also  a 
mlnmax  solution  to  (8)-(10), 

In  the  next  section,  a sufficient  condition 
for  a control  to  be  a mlnmax  control  for  the  prob- 
lem (8) - ( 10)  will  be  presented.  Since  eny  initial 
state  uncertainty  problem  can  be  transformed  to 


then  u (•)  is  a mlnmax  control. 


Proof.  Consider  any  u(*)  and  let  z (•)  be  the 
trajectory  corresponding  to  (u(").qi)  satisfying 


ACCESS": ' k 


V ;l“i  V 

5 -.1  C 


In  Sec.  5, we  show  that  this  suspicion  is  confirmed. 


'v' 


102 


OS  t a M(zi(t),u(t),q1)  + Z X1  (t)k(z1  (t)  ,u(t)  .q1) 
i-1  1 i-1 

•y 

- Z Q M(zi*(t) ,u*(t) .q1) 
i-1  1 

i*  * i i* 

- L \l<t)k(zl  (t),u  (t),ql)+  £ X^z^O-z1  (t)) 

i-1  i-1 

V V 

= Z a M(zi(t) ,u(t) .q1)  - Z a M(zi*(t) ,u*(t) .q1) 
i-1  i-1 

^ d i i i* 

+ t ~ Xl(t)(*l(t)-zx  (t)) 

i-1 

Integrating  the  above  inequality  from  t^  to  t^, 

using  (i)  and  the  fact  that  z*(t  ) * z**(t  ),  the 

o o 

above  inequality  becomes 

y rtf  i i 

Oil  or  M(zl(t),u(t),q  )dt 
i-1  Jt  1 
° t 

y r * i*  * i 

- Z Qf.M(z  (t),u  (t),q  )dt 

i-1  Jt  1 
o 

Y (zi*(t  ),q) 

+ z (z  (t{)  - z (tf))  (11) 

i-1 


From  the  convexity  assumption  and  the  fact  that 
> 0,  i - 1 , • • • , Y 

Y Y 

Z a ; (z1(t  ) .q1)  - I a J (zi#(t  J-q1) 
i-1  11  i-1  1 1 

Y 3*  (z^Ct.l.q1) 

* IOi  57 (z^t^-z1  (tf))  (12) 

Combining  (11)  and  (12)  leads  to 

^fi  i T ^ i il 

L.  or  U(*l(t  M1)  + | M(zl(t),u(t),q  )dt 

i-1  "t 

o 


y r 1*  iP^i*  * il 

a l.  a <l(z  (t  ),q  )+  M(z*  (t)  ,u  (t),q  )dt 

i-1  ' Jt  J 

o 

Since  o > 0,  i - l,***,Yf  there  exists  an 
i«t.l,""*,Y}  such  that 

1 f 

♦ (zi(t ),qS  + f M(zl(t),u(t),q1)dt  2 
^ t 


f 

l*  i P i*  * i 

V (z  (tf)*q  ) + J M(z  (t),u  (t),q  ) dt 


or, 


J(u(-),ql)  2 J(u*(-),q1) 


or, 

sup  J(u(-),q)  2 J(u  (O.qS 
q£Q 

Since  q1eX(u  (•)),  J(u  (•),qi)  - sup  J(u*(-),qi) 

q€Q 

and  the  theorem  is  proved.  □ 

This  theorem  suggests  the  following  method 
for  finding  a minmax  solution  when  there  is  no 
saddle  point  solution. 

1.  Transform  the  initial  state  uncertainty  prob- 
lem into  the  form  (5) -(7). 

2.  Choose  a number  Y 2 2 and  vectors  q^cQ, 
i - 1, • • • ,Y. 

3.  For  this  choice  of  Y and  q*,  consider  the 
optimal  control  problem 

Y Cf 

K(u(-))-  I Q'.r»(zl(t.),ql)+r  M(z1(t),u(t),qi)dt’1, 

i-1  lu  £ J 

o 

z1  ( t ) - kl(zi(t),u(t),q1),  «l(t0)-xfl„.i-l.  — .Y 

where  the  ot ^ are  as  yet  undetermined. 

4.  Use  the  necessary  conditions  for  this  optimal 
control  problem  to  determine  a candidate 
u(t;a'l,.*.,or  ). 

5.  If  possible,  choose  (cr^,  • • • ,cr^)  so  that  for 

all  i, jell, • • • ,y} 

^ f 

|(z1(tf),qi)  + M(z1 (t) ,u(t) ,q1)dt 
O 

Cf 

= ijf  (z^  (t ) ,q^ ) + r M(z^(t)  ,u(t),q^)dt 

* t 

o 

6.  If  such  (Qr^,«»«,or  ) exists  with  > 0, 

y * 

i = 1 , • • • , Y and  la  * 1,  let  u (t)  - 
i-1  1 

u(t  *or  ) and  check  the  sufficient  con- 

^ ’ i 

ditions  of  Theorem  2.  The  functions  X (•)  can 

be  taken  as  the  adjoint  variables  from  the 
problem  in  Step  3.  To  verify  (ii),  one  must 

* i 
evaluate  h(q)  - K(u  (*),q)  and  show  that  q , 

i = 1,«**,Y  maximizes  h(q)  subject  to  C(q)  SO. 

7.  If  no  Qj , i - 1 , • • • ,Y  can  be  found  with  o\  >0, 

Y 

1 - 1,  • • • ,Y  and  I.  a - 1,  return  to 
i-1  i 

Step  2 and  choose  a new  set  (Y,q  ). 

What  makes  this  technique  difficult  to  apply 
is  that  there  is  no  apparent  technique  for  making 

a good  choice  of  Y and  the  vectors  q*  in  Step  2. 

It  may  be  necessary  to  try  many  combinations 
before  the  method  will  be  successful.  Neverthe- 
less, the  method  is  a possible  one  for  finding 
mirunax  controls  and,  except  for  a special  linear 
quadratic  case,  the  only  one  known  to  the  author. 

We  next  apply  this  technique  to  the  simple 
example  of  Sec.  3.  After  applying  the  transfor- 
mation to  the  problem  (5) -(7),  we  have 


mwmmmm 


i r1. 2, 


K(u(0  ,q)  - ^ (z  (1)  + q)  + 2 v u 


dt 


z(t)  - u,  z(o)  = x^,  Q * tq  : Iql  s 1} 


In  Sec.  3,  the  minmax  control  for  x 21  and 

om 


x £ -1  was  found.  Here  we  consider  -l<x  <1. 
om  om 


Since  q is  chosen  to  maximize  K(u(0,q),  we  expect 
q to  be  on  the  boundary  of  Q.  Thus  we  choose  Y=2, 
1 2 

q = 1,  q = -1.  With  this  choice,  the  optimal 
control  problem  in  Step  3 becomes 


K(u(-), a1,a2)  =\ a1(z1(l)  +1)2+|  «1  J u2(t)dt 

w o 


1 2 2 1 \ 2 
+ cr2(z^(l)-l)  c*2  J u (t)dt 


il  = U * *2  “ U * zl^o)  * z2^o) 


From  the  optimal  control  necessary  conditions, 


we  obtain  u(t;ar^,or2) 


O'  -O' 

1 2 1 
- — x + — - — and  then 
2 om  2 


1 + x 


1-x 


from  Step  5,  a 


°2  " 


• Thus , 


the  minmax  candidate  is  u (t)  - - x . Note  that 

om 


this  can  only  be  a candidate  when  x £ 1 since 

om 


u(t)  is  constrained  to  satisfy  |u(t)|  ^ 1. 
Using  Theorem  2 with 


u*(t)  - -x^,  V = 2,  q1  =*  1,  q2  = -1 


x\t) 


1+  x 


- x -1 
2S.  \2(t)-- 25^.  a. 


1+  x 


1-x 


2 » ^ vw  2 » 1 2 1 2 2 

conditions  (i)  and  (iii)  are  readily  satisfied. 

1 
2 


1 2 

Condition  (ii)  is  also  satisfied  since  h(q)  * ^ q 


12  1 2 

+ — x^  and  q » 1,  q = -1  both  maximize  h(q) 


subject  to  |q|  £ 1. 

Combining  this  result  with  that  of  Sec.  3,  we 
can  conclude  that  the  minmax  control  is 


r 


u (t) 


- 1 (*  +1>  if  x * 1 
z om  om 


if  -1  < x <1 
om 


? (x  -1)  if  X £ 1 
i om  om 


The  results  presented  above  are  illustrated 
further  with  the  examples  in  the  next  section. 


4.  Example  Problems 


rl  2, 


Example  1.  J(u(-).q)*^  (x2 (1 ) + x2 (1) ) + I u"(t)dt 

o 


x2,  x<o)-q+*1(W  *2-u,  x2(o)-q2+x 


20m 


0 - lq  : qt  " 1 4 0 , q2  - 1 £ 0] 


While  this  is  a linear  quadratic  problem,  it 
cannot  be  solved  by  the  methods  of  [l]  since  Q is 
not  in  the  form  required  there.  If  q is  known, 
the  optimal  control  is 


u(t,q) 


24(x10m  + 1 l8(x20m  + q2} 


29 


(t-1) 


6(x10m+  ql>  • 10(x20m+  q2) 


29 


and 


12 


J(u(-,q),q)  » Jg  + ql) 


14  , ^ 2 

29  (x20m  + q2 


18 


+ 29  (x10m  + ql)(x20m  + q2)  (13) 


Consider  first  Xj^  * 10,  *20m  = ^ use  t*ie 

procedure  outlined  after  Theorem  1.  The  q which 

o T 1 "1 

maximizes  (13)  subject  to  qeQ  is  q = ^ and 

and  minmax  candidate  is 


u (t) 


o,  462  „ 506 

u(t,q  ^ ~ 29~  1 ~ ~ 


29 


With  this  control, 


h(q)  - J(u  (•) ,q) 


1 ,15  ^ ,2 

2 (29  + q2) 


1 404  2 

+ 2 W + ql  + q2)  + 55,536 


Thi 


s u (•)  and  h(q)  along  with  qj  - 1,  q2  ■ 1, 


231 


29 


253  a . . 462  . » , . 

29~>  8i<t>  » 29“  and  82(t> 


t + satisfy  the  sufficient  conditions  of 
29  29  * 


Theorem  1 for  x,„  = 10,  x,_  - 10. 

462  506  10ln  20m 

t - -Tg—  is  the  minmax  control. 

" xt  c 


Thus  u (t) 


Next  consider  the  case  when  the  measured 


initial  state  values  are  x, 


'10m  l*  X20m 
one  applies  the  Theorem  1 technique,  no  infor- 
mation is  obtained  since  the  solution  obtained 
this  way  fails  to  satisfy  the  sufficient  con- 
ditions of  Theorem  1.  Thus  we  approach  the  prob- 
lem through  Theorem  2.  The  transformed  problem  is 

^(t)  « z2(t)  + q2,z1(o)  - 1,  i2(t)»u(t),  z2(o)-0 


If 


K(u(-),q)m-|r(*1(l)  + q1)2+(z,(l)  + q2)2j+  u2(t)dt 

‘ O 


It  is  expected  that  the  q vector  will  be  on  the 
boundary  of  Q and  thus  we  try  some  combination  of 
the  four  vectors 


r i i 

L i J 


r i 

L-l 


r-i  i 

[ i J 


r-i ' 

-1  J 


l r 1 1 2 - 1 ■> 

We  choose  Y = 2 and  q=Ll  > q “ Lr  ^ 

(Other  possibilities  include  Y * 2,  3 or  4 and  a 
corresponding  number  of  vectors  from  the  above  set 
of  four  vectors.)  For  this  choice,  the  optimal 
control  problem  of  Step  3 in  Sec.  5 is 
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r 


KCuOha^.orj,)  - 3 ^[(z^l)  + l)2  + (z2(l)  + 1)2"J 

+ 2 “l  u2(t)dt  + -|  “2i(y1(l)  - l)2 

21  i r1  2 

+ (y2(l)  - 1)  J + 3 o2  u u (t)dt 

o 

z1(t)  » z2(t)  + 1,  z^  (o)  « 1 ; y^t)  ■ y2(t)-l,  y^oj-l 

z2(ti-u(t)  , z2(o)-0;  y2(t)-u(t)  , y2(o)-0 

Following  Step  4 the  solution  of  this  optimal  con- 
trol problem  Is 

u(t, 0^,03)  - ^ (98o1-32o2-18Qf2-32Q!1or2)(t-l) 

- 35  (4to1-50o2-10a2-50Q1a2) 

99  15 

and  from  Step  5,  03  = Tl4*  *2  ~ Tl4‘  Since  03  >0, 
a2  >(>>0'i+a2*  a mlnmax  candidate  Is 

* , ~(r  99_  15_\  36  34 

u (t)  - u^t,  114,  U4j  19  ' 19 

We  now  apply  the  sufficient  condition  of 

■k  * 

Theorem  2 with  the  above  u (•)  and 

99  15  0 i pin  2 r-n 

1 * 114*  a2  ' 114’  Y ‘ 2’  q =LlJ’q  L - 1 J 

- -s  p'32‘  1 

(114)  [-27,324(t-l)  + 1782  J 

»■«.  • -s  h 1 

(114)  |2700(t-l)  - 3150  J 


1* 

z (t)  - 


M , 

19  c 19 


&-  t3  - & t2 
19  19 


M t2  34 
19  * ' 19  C 1 


It  is  straightforward  to  verify  that  conditions 
(1)  and  (iii)  of  Theorem  2 are  satisfied.  To 

verify  (ii),  J(u  (*)»q)  is  needed. 

2 2 

J(u*(‘),q)  *3  (fg+qj^+qj)  + 2 (‘  H + q2  ) + ,504 

1 rr 

To  verify  (11)  it  must  be  shown  that  q ■ . i and 

2T-11  * _ J 

q - maximize  J(u  ("l.q)  subject  to  q£Q.  This 

can  be  done  by  noting  that  this  nonlinear  program- 
ming problem  satisfies  the  conditions  guaranteeing 
the  existence  of  a solution,  finding  candidates 


from  the  necessary  condition  for  nonlinear  pro- 

Vr 

gramming  problems  and  evaluating  J(u  (*)»q)  for 
these  candidates. 

Thus  ail  the  conditions  of  Theorem  2 are 
* 36  34 

satisfied  and  u (t)  * t - — is  a minmax  solu- 
tion when  x =1,  x„_  * 0. 

om  20m 

This  result  was  obtained  by  guessing  7*2, 

1 fl"’  2 I"  -1 1 

q * i ' » 3 ■ 1 .i  J • **  instead  we  choose 

v o 1 ri"i  2 r*i  1 rn 

V - 2,  q - L ! J » q ■ „1  j °r  q ■ LiJ  > 

2 r - 1 n 

q * +i  j , we  find  that  and  or^  do  not  satisfy 
or^  > 0,  or  > 0,  Qr^  + <>2  * 1.  Thus  the  choice  of 

V and  qi  is  crucial  and  several  choices  may  have 
to  be  tried  before  the  minmax  solution  can  be 
found. 

Suppose  one  assumes  the  measured  value  of  the 
initial  state  is  exact  and  finds  the  optimal  con- 
trol u (•)  under  this  assumption.  Then,  when 
opt 

X10m  * X20m  " 10>  S“P  J(uopt ( ' 5 > q)  " 184'55  "hUe 
q£Q 

the  minmax  value  is  183.59.  When  xir,  * 1 and 

lUm 

x._  * 0,  sup  J(u  ^ ( • ) , q ) - 4.362  while  the  min- 

20ra  qeQ  opt 

max  value  is  3.447.  In  the  latter  case  a reduc- 
tion In  cost  of  over  20°/.  may  be  obtained  by  using 
the  minmax  control  as  opposed  to  using  the  opti- 
mal control  with  the  assumption  that  the  measured 
state  is  exact. 

Example  2.  Consider  a spring  mass  system  where 
values  of  the  spring  constant  lie  in  a known  range 
but  the  exact  value  of  the  constant  Is  unknown. 

A force  Is  applied  to  the  mass  and  the  objective 
Is  to  choose  the  force  to  maximize  the  position  of 
the  mass  at  the  final  time  when  the  system  starts 
from  a known  Initial  state  with  zero  initial 
velocity.  The  optimal  control  problem  is 

J(u ( • ) ,k)  - -x  (tt) 


x^t)  - x2(t)  , Xj^O)  = 3 


x2(t)  - -k  x^ (t ) + u(t)  , x2(o)  = 0 

K - [k:  14kS4l  , U - tu(t)  : u2(t)  - 1 * 0} 

Here  we  have  assumed  the  mass  is  unity,  t^  ■ tt, 

and  k is  the  spring  constant.  An  optimal  control 
★ 

u (•)  is  a control  satisfying  for  all  admissible 

u ( * ) , sup  J(u  ( • ) .k)  * sup  J(u( • ) ik) . If  k - 1, 
keK  ksK 

the  optimal  control  is  u(t)  » +1,  te[0,n].  While 
if  k - 4,  it  is 

-1,  te[0,^) 

“<*>  * TT  , 

,+1,  te(2,TT] 

However  since  k is  unknown,  neither  of  these  con- 
trols has  the  minmax  property.  Since  the  problem 
is  of  the  form  (8) -(10),  Theorem  2 can  be  used. 
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Following  the  procedure  outlined  after  Theorem 
2,  a minmax  candidate  is 


u (t)  = 


-1,  te[0,t1) 

+ 1,  te(t1,rr] 


where  t^  is  defined  by  cos  t^  =*y3  -1, 

Using  this  candidate  in  Theorem  2 with 

V - 2,  k - 1,  k - 4,  cr  - 1 - , a - 

V3  v3 


X2(t)  = 


-a^  sin  t 


- Of ^ cos  2t 


Y~  sin  2t 


it  can  be  shown  that  the  above  u (•)  is  a minmax 
control.  Condition  (ii)  is  the  most  difficult  to 
verify  since  it  required  showing  that  k ™ 1 and 
k = 4 are  solutions  to  the  nonlinear  programming 
problem  of  maximizing 


| cos  Vk  ( t 1*tt)  - (|  + cos 


Vkn  - - 


subject  to  1 ^ k £ 4. 


7.  Concluding  Remarks 

The  optimal  control  of  systems  with  uncertain 
initial  state  measurements  or  with  parameter  uncer- 
tainty in  the  state  equations  has  been  considered. 
The  optimality  criterion  was  taken  to  be  minmax. 

For  problems  with  initial  state  uncertainty, 
the  initial  state  measurement  space  can  be  divided 
into  two  regions.  In  one  region,  the  problem  has 
a saddle  point  while  in  the  other  there  is  no 
saddle  point  solution.  Theorem  1 is  applicable 
when  the  initial  state  measurement  is  in  the  first 
region  and  Theorem  2,  while  applicable  for  both 
regions,  is  more  useful  when  there  is  no  saddle 
point  solution.  At  present,  there  is  no  simple  way 
to  determine  a priori  if  there  is  a saddle  point 
solution  for  the  measured  initial  state  under  con- 
sideration. 

For  situations  where  there  is  no  saddle  point 
solution,  the  initial  state  uncertainty  problem  was 
transformed  into  a problem  with  initial  state  known 
but  with  uncertain  parameters  in  the  state  equa- 
tions. This  allowed  us  to  derive  Theorem  2,  and, 
in  so  doing,  also  obtain  results  which  are  appli- 
cable to  problems  that  are  modeled  with  uncertain 
parameters  in  the  state  equations. 

Re ferences 

1.  Wilson,  D.  J.  and  G.  Leitmann,  "Minmax  control 
of  systems  with  uncertain  state  measurements," 
to  appear  in  J.  of  Applied  Math,  and  Optimiza- 
tion. 


3.  Ragade,  R.  K.  and  I.  G.  Sarma,  "A  game  theo- 
retic approach  to  optimal  control  in  the  pres- 
ence of  uncertainty,"  IEEE  Trans,  on  Automatic 
Control,  Vol.  AC-12,  Aug.  1967,  pp.  395-401. 

4.  Speyer,  J.  L.  and  U.  Shaked,  "Minmax  design 
for  a class  of  linear  quadratic  problems  with 
parameter  uncertainty,"  IEEE  Trans,  on  Auto- 
matic Control,  Vol.  AC-19,  April  1974, 

pp.  158-159. 

5.  Salmon,  D.  M. , "Minmax  controller  design," 

IEEE  Trans,  on  Automatic  Control.  Vol.  AC-13, 
Aug.  1968,  pp.  369-376. 

6.  Pearson,  J.  0.,  "Worst-case  design  subject  to 
linear  parameter  uncertainties,"  IEEE  Trans. 
on  Automatic  Control,  Vol.  AC-20,  Aug.  1975, 
pp.  167-169. 

7.  Blum,  H.  S.,  "Min-max  feedback  control  of 
uncertain  systems,"  in  Differential  Games  and 
Control  Theory  edited  by  E.  Roxin,  P.  T.  Liu, 
and  R.  Srernberg,  Marcel  Dekker,  New  York  1974. 

8.  Gutman,  S.,  Differential  games  and  the  asymp- 
totic behavior  of  linear  dynamical  systems  in 
the  presence  of  bounded  uncertainty,  Ph.D. 
Thesis,  University  of  California,  Berkeley, 
1975. 

9.  Leitmann,  G.  and  W.  Schmitendorf , "Some  suf- 
ficiency conditions  for  Pareto  optimal  con- 
trol, J,  of  Dynamic  Systems,  Measurement  and 
Control , Vol.  95,  Dec.  1973,  pp.  356-361. 

10.  Leitmann,  G. , "A  note  on  a sufficiency  theorem 
for  optimal  control,"  J.  of  Optimization 
Theory  and  Application,  Vol.  3,  1969,  pp. 76-78. 

11.  Stalford,  H.,  "Sufficient  conditions  for  opti- 
mal control  with  state  and  control  con- 
straints," J.  of  Optimization  Theory  and 
Applications  , Vol.  7,  1971,  pp.  118-135. 

12.  Schmitendorf,  W.  E.,  "Differential  games  with- 
out pure  strategy  saddle  point  solutions," 

J.  of  Optimization  Theory  and  Applications, 

Vol.  18,  No.  1,  Jan.  1976. 

13.  Schmitendorf,  W.  E.,  "A  sufficient  condition 
for  minmax  control  with  uncertainty  in  the 
state  equctions,"  IEEE  Trans.  Automatic  Con- 
trol. Vol.  AC-21,  Aug.  1976,  pp.  512-515. 


Dorato,  P.  and  A.  Kestenbaum,  "Application  of 
game  theory  to  the  sensitivity  design  of  opti- 
mal systems,"  IEEE  Trans,  on  Automatic  Control. 
Vol.  AC-12,  Feb.  1967,  pp.  85-87. 


) - 


